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Revealing Student Thinking about Experimental Design and the Roles of 
Control Experiments 


Abstract 

Well-designed ‘ controls” distinguish experimental from non-experimental studies. Surprisingly, we found that 
a high percentage of students had difficulty identifying control experiments even after completing three 
university-level laboratory courses. To address this issue, we designed and ran a revised cell biology lab course 
in which students participated in weekly 4 experimental control exercises.” To measure student understanding 
of control experiments, we developed a set of assessment questions; these were given to students prior to and 
following completion of either a standard cell biology lab course or the revised cell biology lab course. Not 
unexpectedly, the results indicate that the revised course led to greater improvements in students’ ability to 
identify and explain the purpose of control experiments. Based on these observations, we recommend that 
explicit and detailed discussions designed to identify the design and purpose behind control experiments 
become a standard component of all laboratory courses. 
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Abstract 

Well-designed "controls" distinguish experimental from non-experimental studies. 
Surprisingly, we found that a high percentage of students had difficulty identifying control 
experiments even after completing three university-level laboratory courses. To address 
this issue, we designed and ran a revised cell biology lab course in which students 
participated in weekly "experimental control exercises." To measure student understanding 
of control experiments, we developed a set of assessment questions; these were given to 
students prior to and following completion of either a standard cell biology lab course or the 
revised cell biology lab course. Not unexpectedly, the results indicate that the revised 
course led to greater improvements in students' ability to identify and explain the purpose 
of control experiments. Based on these observations, we recommend that explicit and 
detailed discussions designed to identify the design and purpose behind control experiments 
become a standard component of all laboratory courses. 

Keywords: undergraduate, collaborative groups, experimental controls, pre- and post¬ 
responses, science process skills 


Introduction 

A major effort in current science curricular reform involves helping students learn how to 
make informed decisions regarding the significance of scientific observations and 
hypotheses throughout their lives. It is often assumed that "scientific literacy" will be useful 
in this regard and that the acquisition of science process skills, such as those presumed to 
be provided by laboratory courses, is essential to getting students to achieve this goal 
(American Association for the Advancement of Science, 1993; Council of Ministers of 
Education, 1997; Miller, Osborne, & Nott, 1998). A key, and perhaps defining feature of the 
scientific strategy, particularly within the biological sciences, is the role of control 
experiments in both the design and interpretation of experimental observations. Control 
experiments, both positive and negative, are served to explicitly test the experimenter's 
assumptions, so as to validate the conclusions that can be drawn from experimental results. 

Much published science education research has been focused on improving high school 
student science process skills, that is, the development of testable hypotheses, robust 
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experimental design, the critical interpretation of data and subsequent the revision and 
extension of studies (Germann, 1998; Tamir, & Amir, 1987). As part of this work, there has 
been an effort to develop assessments to evaluate such skills at both high school (Burns, 
Okey, & Wise, 1985) and college levels (Brickman, Gormally, Armstrong, & Hallar, 2009). 
Some studies have suggested that the inquiry approach produces greater achievement in 
science process skills than the traditional approach (e.g., Gabel, Rubba, & Franz, 1977; 

Tobin & Capie, 1982; Brickman, Gormally, Armstrong, & Hallar, 2009). Based on these 
studies the authors conclude that basic science process skills can be taught, and that when 
learned, such skills could be readily transferred to new situations. However, there continues 
to be an active debate concerning just what the ability to design, carry out and interpret 
authentic scientific inquiry encompasses, as well as, how to differentiate it from the simple 
inquiry commonly found in high school science classrooms (Chinn & Malhotra, 2002). In 
addition, it is difficult to determine if inquiry-based learning has had a lasting effect 
(Abrams, Southerland, & Silver, 2008; Chinn & Malhotra, 2002). It is arguable that the 
entire peer-review system is based on the fact that rigorous experimental design and data 
interpretation are by no means trivial skills; they must be constantly reinforced. Therefore 
more studies are needed to improve and evaluate students' science process skills at the 
college level. 

Since experimental design and controls are an essential and difficult part of scientific 
process, we hoped to reveal student thinking about control experiments through Ed's tools - 
a web-based program to facilitate the collection, coding, and analysis of text-based student 
data (Klymkowsky & Garvin-Doxas, 2008). In response to the question "It is common to 
hear talk about 'controls' in scientific experiments, what are 'controls' and why are they 
essential part of rigorous experiments?", we find that the use the terms of "positive" or 
"negative" control are rare in student language. The responses below are representative of 
most student responses: 

"Controls are variables that you can make changes to them. They are very 
important to the experiments because they allow you to form a conclusion that could either 
support or reject your hypothesis". 

"Controls are what set the boundaries and allow for the results to be accurate and to 
reduce the change of giving false information or inaccurate results". 

"Controls are ways of double-checking your method. You can check to see if there 
might be another explanation for anything in your test. Each control is one opportunity to 
test something." 

It was in this light that we undertook the present study to learn what students majoring in 
Molecular, Cellular, and Developmental Biology (MCDB) at the University of Colorado know 
about experimental design and controls, and to determine the effect of introducing 
collaborative learning projects on this topic into a one-semester Cell Biology Lab Course 
(CBLC). Collaborative learning has been shown to promote students' conceptual gains, 
(Mazur, 1997; Johnson & Johnson, 1999; Springer, Donovan, & Stanne, 1999 and many 
others), enhance problem solving skills (Cooper, Cox, Nammouz, Case, & Stevens, 2008) 
and boost performance on a wide variety of tasks (Woolley, Chabris, Pentland, , Hashmi, & 
Malone, 2010). 

In this study, we found that students (predominantly sophomores) had difficulty identifying 
and explaining experimental controls after a standard one-semester CBLC, a course 
commonly preceded by two other laboratory courses. These students served as the control 
cohort. Students in this group often confused positive control experiments with 
experiments or negative control experiments. Positive control experiments were often not 


https://doi.org/i0.20429/ijsotl.20li.050208 


2 



IJ-SoTL, Vol. 5 [2011 ], No. 2 , Art. 8 


recognized as necessary. To address these issues, we designed and incorporated a set of 
control exercises into the spring 2010 CBLC; students taking this course served as the 
experimental cohort. For most control exercises, students first worked independently and 
then in collaborative groups to explicitly identify experimental, positive and negative control 
experiments necessary within experiment scenarios, and to explain the purpose of each 
control experiment. As another type of control exercise, students designed experimental 
scenarios based on proposed questions. We present a comparison between these two 
cohorts with regard to assessment gains measured by pre- and post-responses, student 
reasoning on experimental controls and student attitudes. 


Methods 


CBLC Characteristics 

The CBLC is a required 2-credit course taken by MCDB majors. It is also the last lab course 
required of majors; typically these students have already taken two one-credit laboratory 
courses (associated with the Introduction to Cell and Molecular Biology and the Introductory 
Genetics lecture courses). Each section contains between 16 -20 students working in 
groups of 2-4 for 4 hours weekly (one section in spring 2010 contained only 9 students). 

The first half of the semester emphasizes techniques (e.g., microscopy and electrophoretic 
analyses), while the second half focuses on groups of students designing, conducting, 
analyzing, and reporting on their self-designed experiments. Teaching Assistants (TAs) 
work in pairs and each TA is responsible for two sections per week. The instructor meets 
with the TAs weekly to prepare them for the upcoming labs. In spring 2010, the instructor 
(J.P.) and J.S. both met with the TAs to go over the course materials including the control 
exercises. 

Study Participants 

Sophomore or junior students who have completed at least 3 of the 4 MCDB core courses 
and 2 previous lab courses participated in this study. Students (n = 101) who completed 
the standard CBLC without the control intervention at the end of fall semester in 2009 
served as the control cohort. Students (n = 40) who took the spring 2010 CBLC with the 
control intervention served as the experimental cohort. Sixty-five percent and 63% of 
study participants in the control and experimental cohorts respectively took the Molecular 
Biology course concurrently or prior to the CBLC. 

Assessment Questions 

We developed two sets of questions to probe student thinking on experimental design and 
controls (see Appendix 1: Pre/Post Assessment). The first set of questions evaluates 
student understanding of a "negatively controlled experiment" and basic mathematical 
reasoning skills within the context of experimental data (adapted from Stanovich, 2009). 

The second question set represents a typical experimental scenario and evaluates student 
understanding of both the positive and negative control experiments. Within the second 
question set, students took the first four questions (2.1 - 2.4) in the pre-assessment and 
four additional questions (2.5 - 2.8) in the post-assessment. Students were given 15 and 
30 minutes to complete the pre- and post-assessment respectively. Since students were 
not aware that these were assessment questions, some students discussed the questions 
with neighbors in both the pre- and post-assessments. Students first identified the 
experimental conditions in the format of multiple-choice responses [e.g. positive control (P), 
negative control (N), experiment (E) or unnecessary (U)] and then explained the purpose 
for each identified control. 
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Control Exercises 

Our intervention included one online control tutorial, which students completed outside of 
class, and seven in-class control exercises. Six of these seven exercises involved 
identification of positive and negative control experiments and experimental groups within 
an experimental scenario and the generation of conclusions based on these controls (see 
Appendix 2: Example of a Control Exercise). For the seventh exercise, students designed 
an experimental scenario based on provided questions and results. For each in-class control 
exercise, students first worked individually and then in collaborative groups made of 3-4 
students. It took about 20 minutes to complete each exercise. Each exercise was worth 5 
or 10 points upon completion. TAs read through student responses, shared the discovered 
incorrect explanations with one another at the next TA meeting and discussed a set of 
common incorrect explanations with students in the next lab period. 

Data Collection and Analyses 

We collected post-assessment data in the fall 2009 standard CBLC (no intervention), and 
both pre- and post-intervention results in the spring 2010 CBLC. We compared student 
performance in the spring 2010 CBLC by the measure of pre- and post-multiple choice 
selections and pre- and post-written responses to explain the controls. We used ANOVA to 
analyze results for the multiple-choice assessment when three groups were compared. 

When two groups were compared, data were analyzed using both the effect size and 
independent samples t-tests. Effect size was used as a measure of standardized 
differences between two means (Mi and M 2 ) and was calculated as d = (Mi - M 2 )/s, where 
s is standard deviation. An effect size of < 0.4 is generally considered to indicate a small 
effect, 0.4 - 0.8 a medium effect, and > 0.8, a large effect (Cohen, 1988). Two biologists 
who were not involved in the study coded the student written responses for cohesiveness. 

Student Attitudes 

Two attitude survey questions (see Results below) were given to students during the last 
week of the 2010 CBLC. Permission to use student survey data and the pre- and post¬ 
intervention results (exempt status: Protocol no. 0108.9) was obtained from the University 
of Colorado Institutional Review Board. 


Results 

Response to the First Set of Assessment Questions 

The first set of questions presents a common drug trial scenario followed by a series of 
questions (see Question 1 in Appendix 1). The first question (A) provided four categories 
with either a drug treatment or a placebo; students were asked to indicate whether each 
represented an experimental or a control group. The second question (B) asked, "If there 
is a control, what did it control for?" Virtually all students answered these two questions 
correctly on both the pre- and post-assessments. The third question (C) asked: "Is there 
a positive control in this experiment?" Twenty-three percent of students thought that the 
placebo was the positive control on the pre-assessment. Flowever, all students thought 
the placebo was a negative control on the post-assessment, indicating that students had 
learned to recognize a negative control and distinguish it from a positive control. In 
response to the fourth question (D) about whether the drug was effective, 35% of students 
thought the drug was effective on both the pre- and post-intervention assessments. Since 
it requires a mathematical calculation to answer this question and we did not address the 
math issue in this class, we did not expect any change in answering this question between 
the pre- and post-intervention results. Nevertheless, the result indicates that a significant 
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proportion of students relied on impressions rather than calculations to draw conclusions 
about experimental results. 

Response to the Second Set of Assessment Questions 

The second set of questions presents a hypothesis and an experimental scenario (see 
Question 2 in Appendix 1). Figure 1 shows that most students failed to identify the positive 
control experiment on the spring 2010 CBLC pre-assessment. Students also had difficulty in 
identifying the two negative controls, and to a lesser degree, the experimental group on the 
pre-assessment. Students who did not do the control exercises in fall 2009 (i.e., control 
cohort: without intervention) had a mean post-assessment score similar to the mean pre¬ 
assessment score of spring 2010 CBLC students (t-test; p > 0.05 and effect size; d = 0.4 - 
medium). However, the mean post-assessment score for students who completed the 
control exercises (experiment cohort) was significantly higher than the mean pre¬ 
assessment score for the experimental cohorts and the mean post-assessment score for the 
control cohort [(ANOVA; p < 0.001; effect size d = 0.2 - small (i.e., measures the effect 
size between the mean post-assessment scores for the control and experimental cohorts)]. 
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Experimental Conditions 

Figure 1. Comparison of students' pre- and post-assessment performance in spring 2010 
CBLC. The mean post-assessment score is significantly higher than the mean pre-assessment 
for the spring 2010 CBLC students (experimental cohort) and the mean score for the students 
at the end of fall, 2009 CBLC (control cohort; ANOVA; p < 0.001). Control cohort (con) = 
101(without intervention). Experimental cohort = 40 (with intervention). 


One significant caveat to this conclusion (as well as all multiple-choice type instruments) is 
that an analysis of students' written responses revealed that not all students who correctly 
identified the controls via multiple-choice response had coherent or correct explanations in 
the short response section. Figure 2A shows the distribution of student pre-assessment 
choice and explanation for the positive control (Question 2.1). Forty students took both the 
pre- and post-assessments. Seventeen percent of students identified the positive control. 

Of these 17% of students, only 5% provided a correct and coherent explanation, while 10% 
did not provide any explanation and 2% provided non-coherent explanation. The remaining 
83% of students misidentified the positive control as either unnecessary (P = U; 40%), an 
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experimental condition (P = E; 35%), or as a negative control (P = N; 8%). The post¬ 
response for this question (Figure 2B) revealed that not only more students (40%) 
identified the positive control, but all these students also provided correct coherent 
explanations. Moreover, no student mistook the positive control for a negative control 
(P = N; 0%). A higher percentage of students mistook the positive control for an 
experimental group (P = E; 45%). Interestingly, of these 45% of students, 13% of them 
provided an explanation implying that they understood the concept of a positive control but 
were just confusing the terms, not the concept. For example, many students stated that 
the purpose of including this condition in this experiment is the "need to verify the NK cells 
are actually expressing the protein after the viral infection". Fewer students thought that 
the positive control was unnecessary (P = U; 15%); of these 15% of students, 5% of them 
provided an incorrect rationale. An example of these incorrect explanations is: "if a gene is 
placed inside of cells, the protein of interest should be present inside of the cells". This 
suggests that some students failed to answer this question correctly based on a lack of 
content knowledge. 

A. 


correct answer - 
coherent explanation (5%) 

correct answer - 
no explanation (10%) 


correct answer - 
non-coherent explanation (2%) 



B. 



correct answer - 
coherent explanation (40%) 


Figure 2. Pre- and post-responses to assessment Question 2.1 - the positive control. 
(A) Pre-responses; (B) Post-responses 


Figure 3A shows the distribution of student pre-responses to the first negative control 
question (Question 2.2). Twenty two percent of students correctly identified the negative 
control (there were a total of 40 students). Of these 22% of students, 7.5% provided 
correct coherent explanations, 12.5% did not provide any explanations, and one student 
(2.5%) provided non-coherent explanation. The remaining 77.5% students mistook the 
negative control for either a positive control (N = P; 47.5%), unnecessary (N = U; 17.5%) 
or experiment (N = E; 12.5%). The post-response for this question revealed that more 
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students (52.5%) were able to identify the negative control, and most of them (81%) also 
provided correct coherent explanations (Figure 3B). While encouraging, we note that this 
represents barely a majority of the students who can actually provide a coherent 
explanation of a negative control. Figure 4 shows the distribution of pre- and post¬ 
responses for the second negative control question (Question 2.3). Although a similar 
number of students identified this as a negative control on both the pre- and post¬ 
assessments (57.5% and 62.5%), more students provided post explanations (42.5% 
versus 18% for the post- and pre-responses respectively). A significantly higher 
percentage of students (40% versus 7.5%) provided correct coherent explanations on the 
post-assessment. In summary, analysis of student thinking through their written responses 
revealed that the most prominent confusion centered on the difference between a positive 
control and an experiment. Some students also confused a negative with a positive control 
for this experimental scenario. 

A. 


correct answer - 
coherent explanation (7.5%) 


correct answer - 
no explanation (12.5%) 

correct answer - 

non-coherent explanation (2.5%) 



B. 



correct answer - 

coherent explanation (42.5%) 


correct answer - no explanation (7.5%) 
correct answer - 

non-coherent explanation (2.5%) 


Figure 3. Pre- and post-responses to assessment Question 2.2 - the first negative control. 
(A) Pre-responses; (B) Post-responses 


A. 



correct answer - 
coherent explanation (7.5%) 

correct answer - 
no explanation (37.5%) 


correct answer - 

non-coherent explanation (12.5%) 
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correct answer - 
coherent explanation {40%} 

correct answer - 
no explanation (20%) 

correct answer - 

nor-coherent explanation (2.5%) 


Figure 4. Pre- and post-responses to assessment Question 2.3 - the second negative control. 
(A) Pre-responses; (B) Post-responses 


In addition to the four questions that students answered before and after the control 
exercises, four additional questions were given only as a post-assessment in order to 
further probe student thinking (Figure 5). Many students had trouble indentifying the two 
unnecessary experiment conditions, although most students did recognize the third one as 
unnecessary condition. For the first unnecessary condition, a majority of students mistook 
it as a positive control (U = P; >90%). For the second unnecessary condition, a majority of 
students mistook it as a negative control (U = N; >90%). The reason that these two 
conditions were unnecessary is because they refer to a different cell type, and students 
simply ignored this fact (see Question 2 in Appendix 1 for detail). Of the 60% students who 
recognized the additional negative control for this experiment, all of them provided correct 
coherent explanations. This result is consistent with the finding that about 60% students at 
the end of the 2010 spring CBLC course indentified the negative controls (Figure 1). 


100 



unnecessary unnecessary unnecessary negative 


Experimental Conditions 


Figure 5. Post-responses to the additional four assessment questions (2.5 - 2.8). 


We asked if students who took the Molecular Biology lecture course were more prepared to 
correctly answer the control assessment questions than students who had not taken the 
course. We found that there was no statistical difference in student pre-test performance 
between the students who had and had not taken the Molecular Biology course for both the 
fall 2009 control (66 vs. 35 students; t-test; p > 0.05) and the spring 2010 experimental 
cohorts (24 vs. 16 students; t-test; p > 0.05). 
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Survey results 

We asked whether students thought that the various control exercises helped their own 
research projects. Seventy percent of students thought they helped in designing their 
research projects (Figure 6). Sample student quotes are listed below. 

Quotes from "moderate help": 

"The controls are an essential part of biology experiment. Without the control 
exercises, I had no experience identifying these types of things, and I think experience is 
absolutely necessary in order to improve". 

"Control exercises provide practice for planning our experiment". 

Quotes from "little help": 

"The controls for our research project were simple whereas the ones given 
throughout the semester were complex, and at times, difficult to understand". 



■ little help 

■ moderate help 

■ much help 


45% 


Figure 6. Student attitudes on how much the control exercises help their class research 
projects. Most students think that control activities help their class research projects. 


Students had a mixed response to the second survey question, e.g., "rate the difficulty level 
for the various control exercises". They seemed to indicate that it depended on the 
individual control exercise; some were easy, others were difficult. 


Discussion 


Our control exercises were designed to help make students aware that both positive and 
negative controls are necessary when designing an interpretable biological experiment. 
Here we run into some real world complexities; in particular, it is not always possible to 
have both types of controls; positive control experiments can often be quite difficult to 
construct. For example, consider the situation in testing a new drug for a disease. If the 
drug is a small molecule, we can determine the purity and homogeneity of the sample to 
be used in the study, but if the molecule exists in multiple forms (like a protein might) and 
has no easily assayable activity, it can be more difficult to devise a way to standardized 
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experiments (for example, to test the efficacy of a vaccine in a particular group). In this 
light, herbal extracts can be difficult to test if we do not know the concentration of the 
"active ingredient" (assuming that such an active ingredient has been identified). Different 
batches of the "same" substance make a dramatic difference in "potency". Often we are left 
to conclude only that the substance tested and no significantly different effect than the 
negative control. It becomes a skill to even recognize when a positive control is not 
possible. Alternatively, sometimes multiple positive and or negative controls are needed to 
produce meaningful results or to judge the reproducibility of the observed behavior (a topic 
not explicitly considered in this study). How many controls and what types are needed 
within a particular experimental scenario depends on what questions are to be asked and 
what we already know about the particular biological system to be used. 

Based on the poor ability of students to explain positive and negative control experiments, 
even after two introductory level laboratory courses (as well as multiple science courses, 
which one would presume would address these issues explicitly), we developed an 
intervention to help students better analyze experimental design. Our results indicate that, 
after completing eight control exercises in a one-semester cell biology lab course, many 
students improved their the ability to both correctly identify control experiments and to 
offer coherent and correct explanations of the purpose for these controls. In addition, more 
students were able to differentiate a positive control from an experiment or a negative 
control. That said, we are under no illusions that this particular intervention cannot be 
dramatically improved; we only point to the fact that explicitly and repeatedly raising the 
issue of positive and negative control experiments does seem to help. Better interventions 
and more intense and rewarding experiences for students are likely to produce even more 
robust scientific "habits of mind." In this light, it is worth remembering that the 
improvements observed were specific to the context of our intervention. While students 
showed overall improvement on the post-assessment, they failed to show improvement on 
the drug effectiveness for Question 1, which requires mathematical and analytical 
reasoning. Notably, the post-performance of the control cohort (fall 2009 CBLC students) 
was similar to the pre-performance of the spring 2010 CBLC students, and the post¬ 
performance of the spring 2010 students was significantly better than either of these groups 
(Figure 1). 

Our data show that more students recognized the second negative control than the first 
negative control on the pre-assessment. There are two possible explanations. First, we 
investigated whether students who had taken a molecular biology course concurrently or 
prior to the CBLC (65% and 62% of the 2009 fall and 2010 spring CBLC students 
respectively) had an advantage answering the assessment questions though we intended 
to make the assessment questions content-independent. We found no statistical difference 
between the two groups. Second, during the one-semester interaction with students and 
specifically from student pre-assessment written response, we discovered that many 
students thought a negative control was simply adding "nothing" to an experimental 
system. This thinking explains why more students (58%) recognized that infecting cells 
with an "empty" vector was a negative control (the second negative control) than the first 
negative control (22%) on the pre-assessment. 

Student written responses provided insight into student thinking. While some students 
provided coherent explanations when choosing the correct answers this was by no means 
universal; a number of students choose the correct response and provided an incorrect 
justification. This was demonstrated in the response to the second negative control 
condition where a similar number of students identified the negative control on both the 
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pre- and post-assessments, but a significantly higher number of students provided coherent 
explanations on the post-assessment than the pre-assessment. We conclude that we need 
to look at both the multiple choice and especially the written responses to reveal true 
student understanding [an insight we are not the first to have - see (Nehm & Reilly, 2007)]. 
However, not all students provided written responses. There are two possible reasons for 
this. First, since both the pre- and post-assessments were not graded, some students may 
not have made the effort to provide what they thought were unnecessary written responses. 
Second, it is possible that students were more confident and therefore more enthusiastic 
about providing written responses on the post-assessment than on the pre-assessment, 
(e.g., for Question 2.1, 7% and 40% of students provided pre- and post-written responses 
respectively; for Question 2.2, 10% and 45% of students provided pre- and post-written 
responses respectively, and for Question 2.3, 20% and 42.5% of students provided pre- 
and post-written responses respectively). 

It would be informative to measure students' ability to transfer what they learned about 
experimental design through the control exercises to their final research projects. Since 
each student group research project was evaluated based on the overall oral presentations 
by the group members, it was difficult to make correlation of the experimental design and 
the post-control performance for each student. Nevertheless, all research groups except 
one (a group composed of a single student requested to work on his own) included and 
discussed controls in their experimental design. We will evaluate these students as they 
progress to senior year using a capstone or exit assessment that includes questions 
targeting experimental design (in development). 

Teaching students to think experimentally is both important and non-trivial; it is necessary 
for them to objectively and skeptically evaluate the various claims made by many experts. 
Thinking like a scientist should allow them to understand the reach and limitations of 
scientific conclusions. In this study we provided students the opportunity to think explicitly 
about experimental design and why various types of controls are essential in order that 
rigorous conclusions can be drawn. It is clear from our data that knowing how to design 
an experiment, and being familiar with the different roles of controls requires continued 
practice. Our results show a first, albeit modest, successful step towards achieving this 
goal. 

How do we use the results from studies of student thinking to improve our courses and 
curricula? Clearly, the first step is to disseminate our observations as widely as possible, 
starting with our own department. We found that our faculty believe designing control 
experiments is an essential skill, similar to those in other science departments (Coil, 
Wenderoth, Cunningham, & Dirks, 2010). However, these same instructors do not, by their 
own admission, explicitly teach or reinforce such a skill in their courses. Because even 
advanced students struggle with these ideas, we suggest that instruction about controlled 
experiments be explicitly incorporated into every science course. We recommend that for 
every experiment mentioned in lecture, instructors take time to promote student discussion 
of the way measurements were carried out and which experimental controls were required 
to produce rigorous conclusions. This would provide students the opportunities to learn 
about both the various types of controls and their significance in the context of critical past 
studies. Students can then practice such a skill of designing control experiments in every 
lab course. As students learn more about the design, implications, and limitations of 
biological experiments during their undergraduate education we can, at the very least, 
expect to train them in the dispassionate analysis of research results. In this same light, a 
more rigorous introduction into the application of statistical analyses, including effect size, 
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is likely to be helpful in interpreting experimental results. 
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Appendices 


Appendix 1: Pre/Post Assessment 

1. 340 patients with Alzheimer' s disease are enrolled in a study of a new drug. The results 
from the study are: 


Experimental 

□ 

□ 

a 

□ 


Control Category 

□ 200 were given the drug and their symptoms improved 

□ 75 were given the drug and their symptoms did not improve 

□ 50 were given a placebo and their symptoms improved 

□ 15 were given a placebo and their symptoms did not improve 


(A) For each of the four categories above indicate whether it represents an experimental or 
a control group. 


(B) If it is a control group, what did it control for? 
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(C) Is there a positive control group in this experiment? If so, what does it control for. 


(D) Based on these data, was the drug effective? (justify your answer). 


2. To test the hypothesis that high levels of a viral protein (LMP-1) inhibit cell division in NK 
cells, the researchers have: 

1. constructed viral vectors that express either LMP-1 and Green Fluorescent Protein 
(GFP) together or GFP alone in NK cells. 

2. developed an antibody that recognizes LMP-1 protein. 

3. devised a method to efficiently infect NK cells. 

4. devised a method to identify those NK cells that have divided during the 
experimental period. 

For each experimental choice, indicate whether this is a positive control (P), a negative 
control (N), experimental (E), or unnecessary (U). 


For control experiments, explain what they control for. 

2.1 PO N □ E □ U □ Determine whether the LMP-1 protein is present in LMP-l/GFP 

infected NK cells. 

2.2 PO NOEOUO Infect NK cells with the GFP vector and count the number of 

GFP-expressing cells that divided during the experimental 
period. 

2.3 PO NQEQUQ Infect NK cells with an "empty" vector (e.g., expresses neither 

LMP-1 nor GFP), and count the number of virally infected cells 
that divided during the experimental period. 

2.4 PO NQEQUO Infect NK cells with the LMP-l/GFP vector and count the number 

of infected cells that divided during the experimental period. 

2.5 PO NQEQUQ Examine a cell type known to express LMP-1. 

2.6 PQ NQEQUQ Examine a cell type known not to express LMP-1. 

2.7 PO NOEOUO Determine where LMP-1 protein is located in LMP-l/GFP infected 

NK cells 

2.8 PQ NQEQUQ Examine NK cells for the expression of LMP-1 before viral 

infection. 

If the NK cells divided, what are the possible explanations? 
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If the NK cells did not divide, what are the possible explanations? 


Appendix 2 : Example of a Control Exercise 
Introduction 

The DeDecker lab in MCDB studies the function of a previously undescribed Drosophila 
gene, unglued glia (ugg). Ugg mutants are embryonic lethal. Ugg is thought to be a seven 
transmembrane receptor protein. In order to determine what is wrong in homozygous ugg 
mutant embryos, stage 16 embryos were fixed and stained for HRP (which is expressed 
specifically in neurons at this stage) and Repo (which is expressed specifically in glial cells). 
This was the result: 


HRP - Neurons 


Wild type 


Ugg 

mutant 


Repo-Glia HRP-Neurons 

Repo - Glia 




Additional information 

A) Positive control experiments are designed to ensure that reagents work and procedures 
were carried out correctly. 

B) Negative control experiments are designed to check whether experimental outcomes are 
due only to the specific variable being tested. 

C) We have access to a previously characterized mutant, glial cell missing (gem), which 
does not develop detectable glial cells and death with nerves (dwn), in which homozygous 
embryos die during development at the same time as homozygous ugg embryos. 

1. Were positive control(s) included in this experiment? 

□ Yes. The following were positive control(s) and they controlled for 

□ No. What should the experimenters have done? 
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2. Were negative control(s) included in this experiment? 

□ Yes. What is it and what is it controlled for? 

□ No. What could be used as a negative control? 

3. Given the control(s) that were included, what can you conclude from this experiment? 
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